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ABSTRACT 

Increasingly, faculty academics are required to teach and design online courses. However, in many cases, faculty 
members report having low levels of confidence, self-efficacy and competence to teach in online environments. Although 
their professional learning is often enhanced by institutional support strategies such as workshops, online instruction and 
mentoring systems, many faculty academics learn through “just-in-time” rather than “just-in-case” strategies. 

This paper reports on the findings from a cross-continental research project between researchers in two higher education 
institutions in the United States and Australia. The project was initiated to: 1) determine the learning needs of faculty 
members who teach online and design online courses; and 2) to develop tailored professional learning programs and 
resources to enable faculty members to become effective online teachers and skilled online course designers. 

As well as providing an account of the research findings to date, the paper provides recommendations for other 
researchers who may be considering cross-institutional or cross-continental research about online teaching, online course 
design and professional learning programs. 
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1. INTRODUCTION 

1.1 Background and Significance 

The body of literature examining online instruction sheds light on the challenges faced in many higher 
education institutions. Zhen, Garthwait, and Pratt (2008) studied factors influencing 400 faculty members’ 
decisions to employ multiple forms of online course management applications (such as Blackboard and 
WebCT). The researchers found that motivational factors such as personal philosophies and self-efficacy had 
a strong impact on the probability of adopting online course management applications. The lack of clearly 
articulated pedagogy for online instruction presents further challenges to online teachers. Shephard, Madelon, 
Alpert, and Koeller (2007) argue that instructors’ presumed lack of technical competence, knowledge, and 
anxiety contribute to inefficacious beliefs for teaching online. The authors highlight the need for continued 
and varied professional development opportunities to bolster the efficacy of online educators. 

Teachers’ beliefs in their self-efficacy affect their general orientation toward the educational process as 
well as how they design instructional activities (Pajares, 1992; Prosser & Trigwell, 1997; Samuelowicz & 
Bain, 2001; Trigwell & Prosser, 1996). Teachers with a strong sense of self-efficacy are more inclined to 
demonstrate greater levels of planning, organization, eagerness, and allot additional time teaching in areas 
where their sense of self-efficacy is higher, while tending to steer clear of subjects and topics when self- 
efficacy beliefs are lower (Tschannen-Moran & Woolfolk Hoy, 2001). They also tend to be more accepting 
of new ideas, more eager to try new techniques, are more dedicated to teaching, persevere during tumultuous 
endeavors, recover better in light of problems, are less critical of students who make mistakes, and work 
longer with students experiencing difficulty (Gibson & Dembo, 1984). Furthermore, their experiences of 
teaching challenges and “bumpy moments” can catalyze their professional learning (Romano, 2006). These 
findings suggest that fostering teacher self-efficacy is integral to sound educational practices. 
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In light of the challenges surrounding online education, the primary aim of the research is to examine 
online teaching self-efficacy, threshold concepts and troublesome knowledge that are encountered by faculty 
as they develop their online teaching skills. The second aim of the research is to develop a set of research- 
informed guidelines, pedagogies, and practices that will enhance the development of professional training 
programs for online education. Threshold concepts are defined as “core concepts that once understood, 
transform perception of a given subject” (Meyer & Land, 2003a) and troublesome knowledge (Perkins, 2006) 
is knowledge that can cause cognitive conflict as learners compare new ideas with their prior knowledge. The 
literature in this area has detailed several challenges of online education including developing one’s identity 
in a virtual environment and overcoming technology to better facilitate instructional practice (Carmichael, 
2012; Koole, 2010; Savin-Baden, Sinclair, Sanders, & Wind, 2007). 

1.2 Conceptual Framework 

Bandura’s Social Cognitive Theory provides the theoretical lens from which to examine the dimensions 
pertaining to online teaching self-efficacy beliefs. Research has established the close association between 
teacher efficacy and commitment to teaching, adoption of innovations, and use of effective strategies 
(Albion, 2001; Bandura, 1997; Kulinna & Silverman, 2000). Because self-efficacy beliefs are conceptualized 
to be multidimensional and are specific to a particular area or domain, the examination of self-efficacy 
beliefs within online education will extend the theory of self-efficacy to this instructional medium. 

1.3 Specific Aims 

The initial aim of this investigation is to examine the dimensions and perceptions of postsecondary online 
instructors through interdisciplinary and transcultural collaboration. Through this process, the research seeks 
to identify the challenges faculty face in online course development and instruction. The long-term goal of 
the project is to provide empirically-based evidence to inform the customised development of future 
professional learning activities. Through interdepartmental, institutional, and cultural comparisons, we also 
aim to delineate the contextual, or cross-continental, factors that influence teaching and learning. These 
research questions outline the study’s aims: 

1 ) What are the threshold concepts that faculty encounter when they learn about online education? 

2 ) What are the online teaching self-efficacy beliefs of faculty, and do these beliefs converge or depart from 

their identified threshold concepts? 

3) Are differences in online teaching self-efficacy and threshold concepts seen across disciplines and 

academic institutions? 

4 ) How can the identification of online teaching self-efficacy and threshold concepts using diverse samples 

of postsecondary online educators be used to inform future academic development programs? 


2. METHODOLOGY 

Drawing from the work of Northcote and her colleagues, this research seeks to build upon an established 
framework of online teaching threshold concepts and self-efficacy evaluation developed by the study’s 
researchers (Northcote, Reynaud, Beamish, Martin, & Gosselin, 2011). As part of this collaborative effort, 
two of the study’s researchers (Northcote and Gosselin) met in Cooranbong, Australia at Avondale College 
of Higher Education in October of 2012 to administer, collect and analyze the study data. 

2.1 Research Design 

A mixed-methods triangulation design was used for this research (Creswell & Plano Clark, 2007). The design 
involves simultaneous collection and analysis of qualitative and quantitative data throughout the project. The 
study consisted of three steps. The first two steps were conducted concurrently at both researchers’ affiliated 
institutions. The final step integrated these data and provided the cross-continental comparisons of online 
teaching beliefs. 


283 



ISBN: 978-989-8533-18-0 © 2013 lADIS 


2.1.1 Step 1: Reflective Journals and Focus Groups 

The researchers in the study gathered data in the form of systematic reflective journals and focus groups 
approximately two times per month across a four month period. These data provided information about the 
general concerns and breakthroughs reported by online instructors, and were evaluated to identify threshold 
concepts related to online teaching. Findings from these data analyses were examined to determine the 
relatedness to the framework established through earlier research. 

2.1.2 Step 2: Questionnaire and Analysis of Results 

The Online Teaching Self-Efficacy Inventory (Gosselin, 2009) was administered at the time of the final round 
of reflective journaling. Data from this questionnaire provided a means of measuring the degree of 
self-efficacy about faculty members’ online teaching abilities and provided comparative data with the 
qualitative findings from Step 1 . 

2.1.3 Step 3: Formulation of Guidelines for Future Professional Development Activities 

Findings from analyses of data from the reflective journals and focus groups (Step 1) and the questionnaire 
instrument (Step 2) were contrasted with the initial work of Northcote et al. (201 1) to holistically evaluate the 
articulated dimensions on online instruction. Similarities and differences were gleaned from these data by 
comparisons across data collection sites. Finally, these steps informed the development of best practices for 
future professional learning programs in online instruction which augment the programs and resources that 
have been provided to date (for example, Northcote, 2011; Northcote, Reynaud, & Beamish, 2011; 
Northcote, Seddon, & Brown, 201 1). 

2.2 Instruments 

The OTSEl: Online Teaching Self-Efficacy Inventory (Gosselin, 2009) is a five-scale inventory consisting of 
46 total items to assess multiple components of online teaching self-efficacy. Online faculty participants were 
asked to indicate how confident they are in accomplishing the activities by selecting a number for each item 
on a scale ranging from 0 to 10. No confidence is represented by a 0 and complete confidence is indicated by 
a 10. The inventory scales include: (1) Web-Based Course Structure', (2) Online Curricular Alignment, (3) 
Course Content Migration', (4) Virtual Interaction', and (5) Selection of Technological Resources. Alpha 
reliabilities of the scales range from .84 to .95 reflecting excellent internal consistency. The average variance 
accounted for across the five single-factor scales ranges from 45.93% to 64.38% with an average of 53.16% 
of explained variance. The percentage of variance explained for each of the inventory scales provides 
evidence for good factor validity (Stevens, 1996). 

The reflective journal provided the researchers with a format to record their observations of how faculty 
members encountered both successes and challenges in their online teaching and course design processes in a 
typical semester. The reflective journal template incorporated three points of reflection; the first two being 
prompter-questions and the third being a list of key issues, about online teaching; 

• Reflection 1: Erom my point of view, what are the major concerns or areas of “troublesome 
knowledge” that faculty talk to me about or that 1 observe? 

• Reflection 2: What typical questions do faculty members ask me or others about online learning? 

• Reflection 3: Do the faculty ask about or comment on the following concepts? (for example: the 
nature of the online learning environment; student attention; online communication) 

The focus groups aimed to extrapolate additional challenges, concerns, and perspectives surrounding 
online teaching and course development. The semi-structured format consisted of nine questions to examine 
the following: 1) areas of concern in online education; 2) perceptions of students’ concerns in the online 
environment; 3) the nature of online pedagogy in contrast to face-to-face instruction; and 4) experiences 
interacting with colleagues, support staff, and administration in developing courses and teaching online. 
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2.3 Sample 

The population for the study includes faculty currently teaching and/or developing online courses at The 
University of Texas at Tyler (UTT) and at Avondale College of Higher Education (ACHE) in New South 
Wales, Australia. Eor the purposes of this investigation, online courses must have 50% or more of the content 
delivered through online mediums. Twenty participants from each site were solicited for participation in the 
qualitative aspects of this research. The accessible population for the quantitative phase includes 311 (273 
from UTT and 38 from ACHE) faculty currently teaching undergraduate or graduate courses online. A total 
of A = 121 (38.91%) completed and returned the OTSEI for inclusion in the analysis of data. 

2.4 Data Analysis 

Content analysis was conducted to evaluate the reflective journals and focus group responses (see Step 1 
above). Descriptive data from the Online Teaching Self-Efficacy Inventory (OTSEI) (see Step 2 above) was 
analyzed and contrasted with the qualitative findings. Additionally, data across partnering institutions and 
departments have been compared for potential differences and similarities. Outcomes of these data analyses 
were used to inform the planning process for future professional learning programs. 


3. FINDINGS TO DATE 

3.1 Avondale College of Higher Education 

Preliminary analyses of the data gathered in the reflective journals which were kept by researchers at 
Avondale College of Higher Education have been conducted. The data recorded the concerns expressed and 
the typical questions asked by faculty members about online teaching and online course design. Eindings 
from these analyses indicate that, overall, faculty are mainly concerned about learners and the quality of 
learning. When describing how faculty members were putting their ideas about online education into 
practice, words such as “tension”, “suspicion”, “doubt” and “experimentation” were mentioned to describe 
the transition that many faculty members were experiencing between teaching in on-campus contexts to 
online environments. Because Moodle is used as the primary learning management system (LMS), comments 
about understanding how Moodle works is also a focus of many of their comments. 

As part of their data gathering methods, the researchers in this study kept records of the questions that 
faculty members asked about online teaching, online course design and online learning. Many of these 
questions were characterised by references to online enrolment systems, technical functions of the LMS, 
interactive and communication facilities of the LMS and uploading materials to the online courses. 

The major concerns related to areas of “troublesome knowledge” (Perkins, 2006) and threshold concepts 
(Meyer & Land, 2003b) that faculty members reported to the researchers of this study reflected a tension 
between knowing what was possible in online teaching and having the pedagogical and technical skills to 
apply these ideas to practice. Eaculty members made requests for increased professional learning 
opportunities and more streamlined enrolment and support systems. 

3.2 University of Texas at Tyler 

The preliminary results from the University of Texas at Tyler sample resulted in the articulation of challenges 
in five main areas including: 1) technology; 2) design; 3) time; 4) student interaction; and 5) support. The 
comments made by faculty were eclectic in nature and included: a desire to emulate face-to-face interaction 
and to connect with students online; challenges with incorporation of new technology; difficulty reaching 
students that may prefer to remain distant; the need to go beyond expectations to make the content interesting 
to engage students in the online environment; and that online education is much more time intensive on the 
both the development and teaching than traditional, face-to-face courses. In regard to support, the instructors 
expressed that while they felt comfortable should they be required to develop courses without support, having 
instructional design and technological support staff was needed to have a “...refined, polished, and pretty” 
layout for the courses. 
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Support staff mentioned that adoption and integration of new technology was a primary challenge for 
faculty as well as students. Additionally, the support staff perceived that both faculty and students 
underestimated the amount of time it takes to learn new technology. This caused an exorbitant amount of 
pressure from both faculty and students at the onset of each semester due to requests for assistance in a 
concentrated time period. 

Both faculty and support staff expressed that asynchronous interaction allowed for time to think before 
responding, and the nature of these discussions were perceived as richer than what might be seen in face-to- 
face classes. Both groups also felt that the administration was not aware of the vast amount of time required 
to develop and teach online courses. The disconnect was thought to have come about due to a general 
misrepresentation of what online teaching entailed; the participants expressed concern that those 
administrators having little to no experience in online education were unaware of the time, effort, and 
challenges it takes to develop quality online courses and educational experiences. 

3.3 Online Teaching Self-Efficacy 

Table 1 provides the means and standard deviations across the UTT (n = 67) and ACHE (n = 54) samples on 
the five OTSEI scales. On average, the UTT faculty participants rated themselves as least efficacious on the 
Web-Based Course Structure scale (M = 6.71, SD = 1.54) while the ACHE faculty group reported the lowest 
self-efficacy for the Selection of Technological Resources scale (M = 4.00, SD = 2.13). The Selection of 
Technological Resources Scale examines online teachers’ self-efficacy in their ability to select, utilize and 
determine the appropriateness of technology to enhance student learning and enrich instruction. The Web 
Based Unit Structure scale assesses self-efficacy beliefs that comprise the ability to construct and design 
online units that include clear organizational structure, facilitates straightforward navigation and 
communication guidelines, and is aligned with an institution’s mission. Both of these scales involve the 
construction and design of online courses, and interestingly, both groups were the least efficacious in 
carrying out these aspects of online education. Conversely, both the UTT group (M = 7.65, SD = 1.05) and 
ACHE group (M = 5.92, SD = 2.26) reported the highest self-efficacy appraisals on the Online Curricular 
Alignment scale. The scale encompasses faculty’s self-efficacy beliefs in their ability to effectively align 
learning objectives, course assignments, assessment strategies, and learning activities within online courses. 

Table 1. Means, Standard Deviations, and Comparisons across Data Sites for the OTSEI Scales 


Selection of Technological Resources 

M 

SD 

f(119) 

Cohen ’s d 

95% Cl 

ACHE" 

4.00 

2.13 

8.39* 

1.53 

[1.12, 1.94] 

UTT*’ 

6.80 

1.54 




Virtual Interaction 






ACHE 

5.44 

2.16 

6.06* 

1.11 

[0.73, 1.50] 

UTT 

7.32 

1.19 




Course Content Migration 






ACHE 

5.28 

1.95 

7.51* 

1.37 

[0.98, 1.77] 

UTT 

7.48 

1.25 




Online Curricular Alignment 






ACHE 

5.92 

2.26 

5.57* 

0.14 

[-0.22, 0.49] 

UTT 

7.65 

1.05 




Web Based Course Structure 






ACHE 

5.33 

2.09 

4.20* 

0.10 

[-0.26, 0.46] 

UTT 

6.71 

1.54 





n = 54; \ = bl\*p< .001 
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Comparisons were made using independent measures t-tests across the five OTSEI scales for each faculty 
group. The results of the independent samples t-test indicated significant differences across all OTSEI 
subscales {p < .001) with the UTT group reporting higher self-efficacy beliefs across all of the inventory 
scales. Additionally, the UTT group had taught significantly more semesters online (M = 5.37, SD = 5.13) 
compared with the ACHE group (M = 3.00, SD = 3.49). The results of an independent measures t-test 
demonstrated a significance difference in the number of semesters taught online by each group, f(l 19) = 2.88, 
p = .005. Although the UTT sample had spent more time teaching online, they reported less total years 
teaching in higher education (M = 8.04, SD = 5.29) than the ACHE group (M = 11.97, SD = 9.36). The 
results of the independent t-test to assess the differences in total years teaching in higher education was 
significant, f(l 19) = 4.20, p < .001 . 

At this point in the research, comparisons of the combined OTSEI data show significant differences. 
Specifically, across all scales, the sample from UTT reported higher levels of online teaching self-efficacy. 
When evaluating the number of years that each sample had taught online, the UTT sample demonstrated 
significantly more experience in online instruction. According to Bandura’s theoretical conception for the 
formation of self-efficacy beliefs, the primary development of one’s self-efficacy beliefs is through enactive 
mastery experiences (Bandura, 1997). One possibility for the differences in groups in that the UTT sample 
had spent more time teaching online, and as such, we would expect them to have developed higher self- 
efficacy beliefs through proportionally more experience in online instruction. 

3.4 Recommendations for Online Training Programs 

The findings of the research so far, including outcomes of the analyses of data from questionnaires, reflective 
journals and focus groups, have been considered holistically to inform the future professional learning 
activities that are planned for each institution. The evidence from this research indicates that faculty members 
require professional learning opportunities that enable them to extend their pedagogical and technical skills 
in online teaching and course design. 

To meet the needs of the faculty members who participated in this study, the following professional 
learning activities are currently being planned and implemented: 

• Online and printed resource development to provide faculty with self-paced instructional 
materials to extend their online teaching and online course design skills. An example would include the 
development of an online tutorial for instructors that outlines the process for using advanced features of 
lecture recording software. 

• On-campus and online workshops to guide faculty in their development of online teaching and 
course design skills such as workshops focused on the design of learning activities and assessment. 

• Templates and examples of online courses are being developed and sourced to demonstrate 
effective examples, protocols and formats used in effective online courses and online teaching methods. 
Seasoned online educators are currently developing templates that newer instructors may use in guiding 
the development of their own courses. 

• Online education showcases to feature online courses being taught by faculty within each 
institution and across institutions. A rotating system of cross-departmental workshops using online 
educational delivery systems is being considered to allow for exposure of varied and eclectic 
instructional approaches. 

• Research about online teaching to continue to provide an evidence-based approach to inform the 
development of future professional learning programs. With the dynamic and shifting nature of online 
teaching technologies and delivery systems, it is recommended that ongoing, cyclical evaluation be 
carried out within departments and institutions to promote evidence based support to facilitate best- 
practices in online education. 

• Direct involvement of administrators with faculty and staff to inform the allocation of resources 
and support in the successful development and teaching of online courses and programs. It is 
recommended that department chairs, deans, provosts, and other administrators be included in online 
teaching workshops and program evaluation to demonstrate the need for support of online educators. 
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4. CONCLUSIONS 

The findings of this research to date indicate the close relationship between the extent of experience that 
faculty members have teaching in the online environment and their self-efficacy levels. Although differences 
in online teaching self-efficacy were seen across institutions, analysis of the qualitative data showed that each 
group faced similar areas of concerns for teaching and developing courses in the online environment. This 
research has also provided evidence of the close beliefs-practice link between teachers’ ideas about teaching 
and their practical approaches to teaching within the context of online education. 

This investigation employs a mixed methods design triangulation design, and consequently gives a voice 
and contextual understanding of the challenges related to online education within the broader theoretical 
framework of social cognitive theory. By examining both the contextual factors and self-efficacy beliefs of 
online educators, a holistic understanding of the challenges related to online teaching is uncovered that better 
informs the development of tailored professional training programs compared to using singular approaches. 

The findings that emerged from this research study indicate that the concerns of the faculty members who 
participated in this study were primarily related to learning and learners. The faculty members also showed a 
“big picture” understanding of where online education was placed within their institution’s operations and, as 
such, expressed concern that the systems that supported online education (for example, enrolment, IT, student 
services) should support and not hinder the development and implementation of online courses. 

The benefits of the cross-continental research collaboration that has formed the basis of this project were 
many and varied. Lessons learned in each institution were shared, as were resources and research 
instruments. Although separated by vast geographical distances, the challenges and successes of the faculty 
members in both institutions are remarkably similar; both sets of faculty members were concerned about 
pedagogical issues and technical issues, and they both expressed a need for institutional systems so support 
online education. The main difference between the groups of faculty in each institution was their length of 
experience in online teaching and, consequently, their online teaching efficacy levels. Although this 
investigation found many similarities across instructional faculties within institutions in Australia and the 
United States, future investigations are needed to delineate the nature of this relationship across additional 
educational systems that are varied by country and associated institutional culture. The researchers 
recommend replication of the methodology used in this investigation as a cost-effective and comprehensive 
approach from which future investigations may be based. Through this process, the external validity of the 
results may be generalized to other populations. 

The research instruments used in this study, the OTSEI: Online Teaching Self-Efficacy Inventory, the 
focus group questions and self-reflection journals, were relevant to both institutions and faculty members 
across a variety of disciplines. Only minor adjustments were required across the two countries to ensure the 
language used was relevant to local faculty members. Sharing these instruments increased the efficiency of 
the research timeframe and overall processes. These instruments could be used by researchers in other 
institutions to pursue similar investigations about online teaching. 

The findings from this research have reinforced the need for professional learning programs to be tailored 
to suit faculty members’ needs, rather than offering professional development activities that might work or 
that faculty members might need. Instead, the researchers have found that “just-in-time” professional learning 
programs are more appreciated and accessed by faculty than those offered on a “just-in-case” basis. 
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